Adding cleavage benchmarks#404

Open

vue1999 wants to merge 30 commits intoddmms:mainfrom

vue1999:cleavage_benchmark

vue1999 commented Mar 4, 2026

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.

Summary

Adding a benchmark to evaluate the accuracy of predicting cleavage energies of crystalline surfaces.

Linked issue

Resolves #403

Progress

Calculations
Analysis
Application
Documentation

Testing

MACE omat, ORB v3

New decorators/callbacks

no


          Adding cleavage benchmarks

7011b8d

ElliottKasoar added the new benchmark label

joehart2001 reviewed

View reviewed changes

docs/source/user_guide/benchmarks/surfaces.rst Outdated

+              Computational cost
+              ------------------
+              Medium: benchmark involves only single-point calculations, but for 36,718 slab-bulk pairs.

Collaborator

joehart2001 Mar 20, 2026

it would be good to give a rough indication of time e.g. hours on gpu and minutes on gpu?

Author

vue1999 Mar 25, 2026

Yes sorry, on GPU it's roughly 5–20 mins, and a few hours on CPUs.

joehart2001 reviewed

View reviewed changes

ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py Outdated Show resolved Hide resolved

joehart2001 reviewed

View reviewed changes

ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py Outdated Show resolved Hide resolved

joehart2001 reviewed

View reviewed changes

ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py Outdated Show resolved Hide resolved

joehart2001 reviewed

View reviewed changes

ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py Outdated Show resolved Hide resolved

Collaborator

joehart2001 commented Mar 22, 2026 •

edited

Loading

Hey @vue1999, thanks for the PR and its looking super good. A few things:

ive made some code suggestions to the calc script so we save files in the generalised format. I know there are 30,000 files... but what do you think @ElliottKasoar?
ive made some knock on suggestions to the analysis based on these calc changes
ive also made suggestions which implement the density scatter plot + structure visualisation, as theres a lot of structures
We are also going to in the future add the ability to swithc betwen types of errors e.g. mae and rmse, so ive suggested we just keep mae for now unless you're against this?

Let me know if you've got any ideas or are unsure about any of my suggestions, thanks!

joehart2001 reviewed

View reviewed changes

ml_peg/analysis/surfaces/cleavage_energy/metrics.yml Outdated Show resolved Hide resolved

joehart2001 reviewed

View reviewed changes

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

joehart2001 reviewed

View reviewed changes

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py Outdated Show resolved Hide resolved

ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py Outdated Show resolved Hide resolved

vue1999 and others added 19 commits

March 25, 2026 11:42


          Update ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py

aa13732

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py

2de40cd

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py

b7bf3f5

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/calcs/surfaces/cleavage_energy/calc_cleavage_energy.py

22a2d68

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/metrics.yml

c0b5256

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

acc02ef

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

9727b82

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

c988422

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

44a633f

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

51e3400

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py

152e174

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

fcaa3f8

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py

d955056

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py

f2bb3ca

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py

179592e

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/app/surfaces/cleavage_energy/app_cleavage_energy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

7e8ddaa

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

a3c85b0

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

0eee54f

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>

vue1999 and others added 9 commits

March 25, 2026 11:51


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

a8abb46

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

5dc9eb0

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

8f9cc86

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

20f76be

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

b106bd0

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

43ff312

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

708a4cc

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

3c2230f

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>


          Update ml_peg/analysis/surfaces/cleavage_energy/analyse_cleavage_ener…

28da106

…gy.py

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>

Author

vue1999 commented Mar 25, 2026

Thanks for the review @joehart2001! Happy to accept all suggestions and fine to keep just MAE for now.

joehart2001 reviewed

View reviewed changes

docs/source/user_guide/benchmarks/surfaces.rst Outdated Show resolved Hide resolved


          Update docs/source/user_guide/benchmarks/surfaces.rst

96daa54

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels